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Evaluation Brief: 
Implementation and Outcomes 
of Kansas Multi-Tier System 
of Supports: 2011-2014 k 

by Kristin Reedy and Natalie Lacireno-Paqnet 


The focus of the Kansas Multi-Tier System of Supports (MTSS) is to improve outcomes for all students 
by instituting system-level change across the classroom, school, district, and state. Such systemic 
change is accomplished through development of a coherent continuum of evidence-based, systemwide 
practices to support a rapid response to each student’s academic and behavioral needs, and features 
frequent data-based monitoring for instructional decision making. 


Kansas MTSS integrates what is known about 
the components of effective schools into a single 
framework for improved social and academic out- 
comes for all students. Core components include 
evidence-based curriculum, high-quality instruc- 
tion, a comprehensive assessment system, data- 
based decision making, effective intervention, 
fidelity of implementation, ongoing professional 
development, and leadership within an empower- 
ing school and district culture. 

Implementation of MTSS has grown rapidly in 
Kansas and is a key approach used for turning 
around low-performing schools in the state, with 
more than a third of all public schools implement- 
ing MTSS to date. A recently completed evaluation 
of the effort by WestEd found that Kansas MTSS is 
substantially contributing to improved student out- 
comes at the local level, as well as benehtting teach- 
ers, improving instruction, and supporting better 
school functioning. 

Evaluation Findings 


Who Participates in Kansas MTSS? 

Finding 1: The scope and reach of MTSS as a state- 
wide initiative is well established in Kansas. The 


statewide MTSS “presence” is pervasive. The state 
may be at a “tipping point” in terms of statewide 
scale-up and installation of MTSS across the state. 

At this point in time, 48.0 percent of the 1,472 schools 
in Kansas have participated in some level of formal 
MTSS training. These schools represent 67.0 per- 
cent of the 293 districts across the state, showing 
that approximately two-thirds of public school dis- 
tricts in Kansas have schools that have participated 
in MTSS formal training experiences (2005-2014). 
Eighteen districts have taken a districtwide approach 
to implementing MTSS. About 224,000 (43.0 percent) 
of all Kansas public school students are enrolled 
in schools that have participated in Structuring or 
Implementation training since 2008. 

What Happens in Schools Implementing MTSS? 

Finding 2: Schools are demonstrating the hall- 
marks of strong implementation of the Kansas 
MTSS framework; specifically, they are demon- 
strating strong leadership at the school and district 
level, high-quality core curriculum, instruction, and 
assessment practices, an empowering culture with 
increased teacher collaboration, data use, ongo- 
ing professional development, and the alignment 
and integration of school improvement initiatives. 
The consistent implementation of these practices 
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TABLE 1. Classification of Responding Schools by Stage of Implementation 


Stage of Implementation 

2012 

2013 

2014 

Number 

Percentage 

Number 

Percentage 

Number 

Percentage 

No stage 

77 

11.7 

55 

9.3 

31 

5.6 

Exploration 

266 

40.5 

70 

11.8 

70 

12.6 

Installation 

58 

8.8 

16 

2.7 

6 

1.1 

Initial Implementation 

215 

32.8 

402 

67.9 

402 

72.4 

Full Implementation 

40 

6.1 

49 

8.3 

46 

8.3 

Total Implementers 

579 

88.2 

537 

90.7 

524 

94.4 

Total 

656 

100.0% 

592 

100.0% 

555 

100.0% 


Source: Authors’ analyses of primary data collected. 


FIGURE 1. Percentage of Schools at Various 
Stages of Implementation, 2012 to 2014 
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Source: Authors’ analyses of primary data collected. 


results in a smoothly run system that is achieving 
its intended outcomes. 


reported that they were currently implementing 
MTSS (75.5%), with another 10.7 percent of respon- 
dents planning to implement. About 81.0 percent 
(448) of schools responding to the survey are imple- 
menting MTSS at the initial implementation level 
or higher, and this is likely an underestimate as it 
is based only on those schools responding to the 
survey. The majority are implementing in read- 
ing (84.8%). More than half of respondents are 
implementing in math (56.9%) and 43.7 percent 
of respondents are implementing in behavior (see 
Appendix B for a summary of the 2014 survey find- 
ings). The percentage of schools scoring at the full 
implementation stage remained stable at 8.3 per- 
cent for 2014. A similar pattern of responses was 
reported in 2012 and 2013 (see Table 1 and Figure 1). 


Finding 3: Implementation of MTSS at the school 
level is increasingly consistent with the Kansas 
MTSS framework. Schools are moving from explo- 
ration/learning about MTSS to more advanced 
implementation stages over time. 


What are the benefits of MTSS? 

Finding 4: MTSS is substantially contributing to 
improved student outcomes at the local level as 
well as resulting in benefits to teachers, improved 
instruction, and school functioning. 


The 2014 statewide Kansas MTSS School Survey 
of Effective Instructional Practices yielded results 
that provided a snapshot of MTSS implementation 
in schools across Kansas. Respondents to the survey 


Statewide, most respondents to the Kansas MTSS 
School Survey of Effective Instructional Practices 
reported that to “some extent” or "to a great extent” 
implementing MTSS has had a positive impact on 
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student outcomes: students scoring at benchmark 
(89.5%); students scoring as proficient on the state 
assessment (70.3%); a decrease in Office Discipline 
Referrals (77.1%); and a decrease in special educa- 
tion referrals (63.4%). 

Universal screening assessment data from a sample 
of schools at full implementation shows gains on 
average for reading and math (except third-grade 
math), with the largest gains evidenced in kinder- 
garten (see Figure 2). 

Fiowever, hard data on student achievement/ 
improvement are not available statewide. State 
assessment data are inconclusive as to any patterns 
or trends in the achievement levels of schools imple- 
menting MTSS. 

In survey data and interviews with the Core Team 
and case study sites, respondents report benefits 
to students, teachers, and schools as a result of 
implementation of MTSS. Survey responses show 
that schools are observing more students scoring at 
benchmark on the universal screeners, behavior is 
improving, there are fewer special education refer- 
rals, and student engagement and motivation has 
improved. Focus group and interview data indicate 
that schools and teachers are changing their atti- 
tudes and practices related to collaboration and 
shared responsibility for all students, developing a 
common language, and using data in their decision- 
making processes. MTSS helps schools to focus on 
strengthening the core curriculum and instruction 
as well as on interventions. 

How Does the Kansas State Department of 
Education (KSDE) Support and Sustain the 
Implementation of Kansas MTSS? 

Finding 5: KSDE has established an effective 
statewide infrastructure to support and sustain 
MTSS implementation. 

Components include the MTSS Core Team, the cadre 
of Recognized MTSS Facilitators, the annual MTSS 
Symposium, an extensive and frequently updated 
website of materials, research, and resources 
on MTSS, and an ongoing dissemination plan 


FIGURE 2. Change in the Percentage of Students 
at Benchmark, Fall to Spring, 2013/2014 
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Source: Authors’ analyses of primary data collected. 


implemented in coordination with the Technical 
Assistance System Network (TASN). Professional 
development materials are well aligned with 
national professional learning standards, such as 
the Learning Forward standards (2012). 

Evidence from the 2014 school survey, Core Team, 
KSDE leadership, case study principal and admin- 
istrator interviews and focus groups, the document 
review, and the MTSS website demonstrate that 
KSDE and the Core Team are effectively supporting 
implementation of MTSS with fidelity. Specifically, 
respondents to the 2014 school survey reported 
that, “to some extent” or “to a great extent,” KSDE 
has established the necessary infrastructure to sus- 
tain and extend MTSS implementation over time 
(77.4%), that KSDE effectively disseminates infor- 
mation about MTSS (75.9%), that there are sufficient 
resources (76.0%), and that MTSS is clearly aligned 
with other state and local improvement initiatives 
(79.3%). Sixty-one percent of respondents agreed 
that there are sufficient MTSS Facilitators to sup- 
port MTSS statewide. 

How Has Kansas MTSS Implementation 
Changed Over Time? 

Finding 6: MTSS implementation is being refined 
in Kansas to focus on the district rather than 
the individual school level, and to the use of an 
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“integrated model” where the areas of reading, 
math, and behavior are approached simultaneously. 

The shift to investment at the district level using 
the integrated MTSS — a comprehensive, integrated 
three-tier (MTSS: CI3T) framework — will require 
increased capacity on the part of the Core Team. 
According to the 2014 school survey, a slight major- 
ity (56.1%) of respondents described MTSS as a 
“district-led” initiative; however, this does not neces- 
sarily mean that the district has taken a districtwide 
approach (as Wichita has done). Sixty-seven percent 
of Kansas districts have had schools who have par- 
ticipated in formal MTSS training, according to the 
data on annual training participation, while 18 dis- 
tricts have engaged in districtwide training and are 
taking a districtwide approach to MTSS. 

What are the Next Steps for MTSS in Kansas? 

Next steps for MTSS in Kansas may be summed 
up as, “Stay the course.” Investment in district-level 
capacity building, the strengthening of the Core 
Team, and continual adherence to the MTSS frame- 
work, as more and more districts and schools join 
the MTSS ranks, will sustain and expand MTSS over 
time. Investment in a statewide data system would 
enable KSDE to document the impact MTSS is hav- 
ing on student outcomes. 

The evaluation activities over the past four years — 
from the annual online survey to the school case 
studies, the in-depth study of Wichita as the evalu- 
ation’s districtwide example to interviews with the 
Core Team — have provided a valuable perspec- 
tive on sustainability of a statewide initiative such 
as MTSS — what it takes and what the challenges 
are. Staff buy-in and support, the integration and 
institutionalization of MTSS practices so that it 
becomes routine, the “way of doing things” and the 
umbrella for all school improvement efforts appear 
to be key. Training of staff, flexible scheduling, and 
the purchase of resource materials are all neces- 
sary but not sufficient. Sustained and continued 
district and school leadership support for MTSS 
through words and action are critical for statewide 
and local sustainability. 


Kansas is both at a turning point and tipping point 
with regard to MTSS. The shift to a district-level 
focus for training and support and the move to the 
integrated model represent key decision-points in 
the process of statewide implementation. A scaling 
up tipping point may have been reached in terms 
of the number of schools and districts that are 
engaged in MTSS statewide. At this point in time, 
48.0 percent of the 1,472 schools in Kansas have 
participated in some level of formal MTSS train- 
ing. These schools represent 67.0 percent of the 293 
districts across the state. 

A stay-the-course/on-message approach from the 
state level on down appears to be the way to move 
forward with expansion and sustainability. 

About the Evaluation 


After a competitive request for proposal process, the 
Kansas State Department of Education (KSDE) con- 
tracted with WestEd, an independent, not-for-profit 
research, evaluation, technical assistance, and pro- 
fessional development organization, to conduct an 
external evaluation of the Kansas Multi-Tier System 
of Supports (MTSS). The WestEd evaluation team’s 
task was to design, pilot, refine, and implement an 
evaluation system that measures the statewide prog- 
ress of MTSS toward its main goal: creating a state- 
wide system of support to local schools and districts 
in order to increase school capacity to use resources 
in ways that enable every child to be successful. The 
project provided formative and summative evalu- 
ation information to KSDE for improving and sus- 
taining MTSS at the school, district, and state levels. 
Leadership and support from the KSDE Office of the 
Commissioner has been instrumental in the prom- 
ulgation of MTSS as a framework for improving the 
education of all students. 

The evaluation was guided by five evaluation 
questions: 

Scope: How many schools and districts are partici- 
pating in MTSS? 
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Implementation: Annually, how many schools 
and districts are (a) exploring the use of MTSS to 
meet students’ academic and behavioral needs, 

(b) adopting and installing components of MTSS 
(e.g., assessments, curriculum, instruction, etc.), or 

(c) successfully implementing MTSS with fidelity? 

Student Outcomes: How are students in schools 
and districts that are fully implementing MTSS 
performing? 

Statewide System and Infrastructure: How 

effective are KSDE and MTSS Core Team activities 
in supporting statewide implementation of MTSS 
with fidelity by schools and districts? 

Sustainability: How successful are schools and 
districts in sustaining MTSS? 

WestEd designed a multi-year, mixed-methods 
evaluation system. The evaluation system was 
piloted and refined in 2011 prior to full implemen- 
tation in 2012 through 2014. Key data collection 
activities included: 

» An annual survey, entitled the Kansas MTSS 
School Survey of Effective Instructional 
Practices, with responses from more than 500 
schools per year; 

» Two-day site visits to six schools where core 
and intervention instruction and team meet- 
ings were observed, and interviews and focus 
groups with leadership teams, teachers, and staff 
were conducted; 

» A multi-day site visit to Wichita Public Schools 
that included interviews and/or focus groups 
with the Superintendent, leadership teams, the 
MTSS advisory team, as well as day-long visits to 
five schools (2 high schools, 1 middle school, and 
2 elementary schools). Multiple cross-district 
focus groups were conducted with teachers, 
principals, and school-based MTSS Facilitators/ 
coaches; 

» A follow-up visit to Wichita Public Schools that 
included interviews with the Superintendent, a 
group interview with the leadership team, and 
interviews with over ten focus groups with more 
than 100 teachers, coaches, and principals; 
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» Periodic interviews with KSDE leadership; 

» Periodic interviews with all members of the 
Core Team; 

» Focus groups with Recognized MTSS Facilitators; 
» Document review; 

» Annual observation and participation in the 
Kansas MTSS Symposium; 

» Annual collection of grade-level universal screen- 
ing data for selected schools; and 

» Annual analysis of school-level state assessment 
data. 

The evaluation system was designed to give a 
complete picture of the implementation and early 
impact of MTSS. A mixed-methods design was 
used to provide a full and comprehensive view of 
MTSS implementation across Kansas from the per- 
spectives of key stakeholders and participants and 
to offer an in-depth portrait of implementation in 
selected schools and districts across the state. 
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